Electronic patient-reported outcomes in clinical kidney practice (ePRO Kidney): a process evaluation of educational support for clinicians

Background: Patient-reported outcomes (PROs) are increasingly mandated in kidney care to incorporate patients’ perspectives. Objectives: We assessed whether educational support for clinicians using electronic (e)PROs could enhance person-centered care. Design: A process evaluation, using a mixed methods longitudinal comparative concurrent design was undertaken of educational support to clinicians on routine use of ePROs. In two urban home dialysis clinics in Alberta, Canada, patients completed ePROs. At the implementation site, clinicians were provided with ePROs and clinician-oriented education via voluntary workshops. At the non-implementation site, neither were provided. Person-centered care was measured using the Patient Assessment of Chronic Illness Care-20 (PACIC-20). Methods: Longitudinal structural equation models (SEMs) compared change in overall PACIC scores. The interpretive description approach, using thematic analysis of qualitative data, further evaluated processes of implementation. Results: Data were collected from questionnaires completed by 543 patients, 4 workshops, 15 focus groups, and 37 interviews. There was no overall difference in person-centered care throughout the study, including after delivery of workshops. The longitudinal SEMs revealed substantial individual-level variability in overall PACIC trajectories. However, there was no improvement at the implementation site and no difference between the sites during both the pre- and post-workshop periods. Similar results were obtained for each PACIC domain. Qualitative analysis provided insights into why there was no substantial difference between sites: (1) clinicians wanted to see kidney symptoms, not quality of life, (2) workshops were tailored to clinicians’ educational needs, not patients’ needs, and (3) variable use of ePRO data by clinicians. Conclusion: Training clinicians on use of ePROs is complex and likely only part of what is required to enhance person-centered care. Registration: NCT03149328. https://clinicaltrials.gov/ct2/show/NCT03149328


Introduction
Internationally, patient-reported outcomes (PROs) are being collected, at times mandated, in kidney care. 1,2 PRO measures are instruments that patients complete to self-report health outcomes relevant to their quality of life (QOL). 3 They are systematically collected with the hope that patients' responses will inform and guide their care, 1,[4][5][6][7][8] and in turn support person-centered kidney care. 4,[9][10][11][12][13] The global kidney community acknowledges the urgent need for evidenceinformed strategies to improve both the QOL and healthcare experiences of people living with dialysis. 14 Optimal use of PROs at point of care may help address this need.
Use of PRO data to enhance person-centered dialysis care remains largely unexplored, 1 perhaps because routine collection, use, and integration is a complex intervention. Person-centered dialysis care upholds attributes of centeredness, namely that a patient is a person who is unique, and has the right to be heard and to have shared responsibility of their care. 15 Use of PROs is one way of drawing attention to the patients' perspectives of their health outcomes. Decades of research has previously focused on efficiency or effectiveness of PRO data usage to support monitoring symptoms or communication in various populations, 16 including kidney care settings. 1,17,18 However, international evidence-based sources and syntheses have identified that clinicians need training and education to support PRO use in their clinical practice at point of care 2,19-23 (i.e. at the individual level between a person and clinician/s) to enhance person-centered care.
While multidisciplinary kidney practitioners have expressed some support for use of PRO responses at point of care, 6,17,24 they have also identified barriers. For example, they expressed skepticism about the benefits to routine patient care, 24 confused/conflated clinician assessment with PRO self-report, 24,25 perceived responses to address concerns are beyond their scope of practice, 7,26 lacked effective interventions to address concerns, 7,26 worried that PRO collection might increase their workload 6,8,24 and psychological referrals, 24 and feared litigation of negligence if PRO responses were not addressed. 24 These uncertainties have resulted in poor uptake, further identifying the need for a clinician-oriented intervention.
Understanding how to provide educational support to clinicians to optimize use and integration of kidney patients' PRO responses is crucial. The purpose of the study was to assess, from clinicians' and patients' points of view, how personcentered care could be enhanced through clinician-oriented educational support for multidisciplinary kidney clinicians in routinely utilizing electronic PROs (ePROs). The research question was: To what extent do clinicians and patients report improved person-centered kidney care associated with educational support to clinicians on use of ePROs?

Design, complex intervention, sites
This mixed methods study was a process evaluation of educational support to clinicians on routine use of ePROs, which is a complex intervention. 27 We used a longitudinal comparative concurrent design, 28 with clinician and patient focus groups and interviews. The rationale for this approach was to provide contextual understandings and practice implications of the impact of the complex intervention: clinician-oriented educational support for routine use of ePROs. The study was registered. 29 There were two phases over 2 years at two sites. See Figure 1 -Project timeline. The sites included two home dialysis clinics (peritoneal dialysis and home hemodialysis) providing care to all of Alberta, Canada: Northern (implementation site) and Southern (non-implementation site). The two sites were comparable in geographic service to Albertans, dialysis staffing, facilities, and number of home hemodialysis patients (<100); however, the implementation site had more peritoneal dialysis patients overall (>300) than the non-implementation site (>200). At the implementation site, ePRO responses were provided to clinicians; they had collected PROs on paper for 8 years prior to the study. In the non-implementation site, ePRO responses were not provided to clinicians; patients had not previously been invited to complete PROs. In both the sites, patients had online access to their ePRO data, to track responses overtime, share with any of their healthcare team, or use in self-care. journals.sagepub.com/home/taj 3 The Knowledge-To-Action Framework 30 guided both phases of this patient-oriented research. 31 See Figure 2 -Adapted Knowledge-to-Action Framework. Phase I included clinician and patient consultation at the implementation site to develop the clinician-oriented educational support. Following the knowledge-to-action steps, in consultation to determine knowledge gaps of routine use of ePROs, the education support that clinicians endorsed were tailored, co-designed workshops. Workshops included knowledge adapted to the local context, and content that addressed barriers and supports to use of ePROs in kidney care. In phase II, the implementation site was supported via clinician workshops.
Our research team encompassed multi-disciplinary clinicians, decision-makers, researchers, patient organizations, and patients. The Patient Advisory Committee was co-chaired by LL, a Patient Partner, and Advisory members were trained in patient-oriented research and involved throughout the project.

Participant recruitment
Through purposive sampling, 32 all clinicians (nurses, nephrologists, dietitians, social workers) from the implementation site were invited via information sessions, posters, and emails from the department between July 2017 and July 2019.   36 (3) EQ-5D-5L 37 (see Table 1). Internal consistency reliability (based on the ordinal Cronbach's alpha) 38 in this sample was 0.97 for the overall PACIC score, and ranged from 0.82 (delivery system design) to 0.91 (problem-solving) for the domain scores. In phase II, clinicians in the implementation site were asked to anonymously answer two ratingscale questions asking how often they reviewed and used ePRO data to inform their care.
Clinicians were asked to respond to eight questions about their knowledge and beliefs regarding the use of ePROs prior to the first workshop and after all workshops were completed.
Quantitative analysis. We used descriptive statistics to compare the samples at the two sites at the start of the study (time 0) using the Chi-square test for categorical variables and t-test for continuous variables.
To address the research question, we first graphically compared the average linear changes over time for the overall PACIC score and domain scores between the two sites over the 24-month period. We subsequently fit longitudinal structural equation models (SEMs), using Mplus software (version 8.6), 39 to statistically compare average change over time between the two sites. Separate SEMs were fit before and after the workshop start date. The SEMs were specified with two latent factors representing the intercept (outcome score at start of study) and the slope (linear change in outcome scores) over eight repeated measures that occurred at the time of each visit, which were regressed on the grouping variable (site) and the following covariates to control for statistically significant differences between sites: education, ethnic group, and diabetes. Individually varying times of observations relative to Qualitative analysis. Guided by the interpretive description approach, 41 data were analyzed in phase I to guide ePRO provision and inform workshop creation, and in phase II to assess how person-centered care was enhanced (or not) through routine ePRO use. Transcripts from initial interviews or focus groups were listened to and read to create a codebook, iteratively refined during analysis. The data were coded by JC and a trainee under the supervision of KS-M; questions about categorization were resolved through consensus. Managed with N-Vivo TM , data were thematically analyzed for meaning saturation 42,43 to understand diversity of perspectives. Patient and clinician data were analyzed separately, then compared and contrasted. We ensured trustworthiness and rigor by addressing credibility (iterative discussions, triangulation, and negative/alternative case analysis), confirmability (audit trail, field notes), and transferability (context). 41 The Consolidated Criteria for Reporting Qualitative Research guidelines 44 in Supplementary File 2 provides additional details.

Participant demographics
Of the 693 approached patients, 543 (78%) participated and routinely provided ePROs ( clinicians. Forty-one clinicians attended between one and four workshops, and 38 completed questionnaires before and after workshops (Table 3). Figure 3 depicts substantial individual-level variability in overall PACIC trajectories, with greater improvement over time in the non-implementation site relative to the implementation site during the pre-workshop period, whereas the trajectories appear more similar during the workshop period. The results of the SEMs (see Table 4) confirmed that the overall PACIC score was lower at the start of the study for the non-implementation site (intercept of non-implementation = 3.10, p = 0.00; difference in intercepts = 0. 32 Table 3. Demographics of participants in qualitative focus groups and interviews, and clinician workshops. Highest level of education

Demographic information Qualitative focus groups and interviews
Volume 14 10 journals.sagepub.com/home/taj TherapeuTic advances in chronic disease Following process evaluation, we analyzed clinicians' and patients' perspectives at the implementation site for insights into why there was no substantial difference in person-centered care between sites. Insights include: (1) clinicians wanted to see kidney symptoms, not QOL; (2) workshops were tailored to clinicians' educational needs, not patients' needs; and (3) variable use of ePRO data. Illustrative quotes are provided in Table 5, along with exemplars of negative/alternative cases (when applicable) to represent a diversity of perspectives.
Clinicians wanted to see kidney symptoms, not QOL. For 8 years prior to the study, the implementation site had collected the ESAS-r:Renal on

Patients Caregivers
Healthcare providers Physician Allied health Type of dialysis c Peritoneal 20 (13) 2 (0) d 13 (7)  0  15 Home hemodialysis Peritoneal and Home Hemodialysis N/A N/A 11 (6)  14  3 Ethnic background a Clinicians that attended one or more workshops and responded to eight questions about their knowledge and beliefs regarding the use of patient-reported outcomes (PROs) prior to the first workshop and after all workshops were completed. b Phase II is denoted in round brackets. Note that there were eight participants in phase II who were not a part of phase I (eight patients), and 30 participants who were in phase I but not a part of phase II (16 patients and 14 healthcare providers). c For clinicians, type of dialysis in their practice. d Caregiver to a person with this modality.      "If there was a means to attach a trend report to a case summary, like a visual that they could send to the GP, that would be fantastic. I'm not sure if that's doable with our current systems, but -" (Clinician 4, Clinician Focus Group 5, q36) "I don't see the social worker as a good source of mental support." (Patient 1) "Yeah. It's the ah, I was thinking they should refer you to, to speak to somebody in mental health or something, you know." (Patient 2, Patient Focus Group 1, q37)  "I can't say [use of ePROs] has changed. Um, I mean it maybe has made me more aware." (Clinician 2050, q51) "Before the study, I would look at [ePROs] after the fact. And then, the patient had gone home. But nowthey just made us more aware, maybe just to look at it. And address it." (Clinician 2013, q52) "I think my practice has changed a little bit in the sense of it being less data entry and more patientfocused. . .Knowing that the patient's perspective can be highly different than what we usually find when the physician or the RN does the checklist and is clinically prompted. Having the patient's perspective first [due to ePRO completed before appointment] has changed the way that I assess them and the way that I ask questions. I would say that it's developed a little bit more of a trusting relationship in some sense because I can see them as a whole, and having a bit more impact on their quality of life. (Clinician 2011, q53) "I think it's definitely got value . . . in coordinating care and -finding out, you know, the state of our patient -like, 'how are they doing?' Not just, 'are they walking in okay? Do they look okay? Is their blood pressure okay?' It kind of looks at them as more as a person, you know, it's mental health, and social, and physical. It's more holistic, for sure.' (Clinician 2029, q54). "It's just we have to take it one step further, and get the practitioners, or whatever, to change the way they do business a little bit. And also, give the, sort of the, I hate to say the ammunition here but the, ah, to make it real -also, myself responsible for asking questions." (Patient 204, q57).
"The minute I mark whatever number 2 to 10, whatever it is, they should ask me. They should address it . . . ." (Patient 3) "And if they don't, then you go, 'Okay. I guess it was more important to me than it probably should have been.' And you kinda forget about it, right? . . . " (Patient 2) "If I'm reporting something as a four or five, I kinda expect them to, you know, bring it up, so I can talk about it. . ." (Patient 1) "When you get an answer, you can make an assessment for yourself kind of, how serious it is. So when they don't address it, you're kinda left in that kind of limbo, right? Like I'm being maybe more anxious about it than I need to be. . . We're not always equipped; we're not doctors. So, we're not always equipped to know if that is serious, or if that's just a minor inconvenience. So, when you put it down on the sheet, you're expecting them to interpret that information, 'Is this serious? Or is it just something you've gotta live with?' Right? The two opposite extremes." (Patient 2, Patient Focus Group 8, q58).
"I'm hoping that they're going to adopt them and see them as a method of, um, being able to support the patients where they are, so that, um, especially with the ability to see trends and changes, that it will become an automatic part, I should say, it should be a mindful part of the patient and clinician experience.  Routine integration of ePROs as a fundamental change to practice was discussed in Workshop 4. Given ePROs were often not seen as important or useful (q39-41), attendees discussed whether this information could provide a 'missing part in our practice' (q41), and optimal ways to integrate ePROs into workflow (q39, [42][43][44]. 'Contradictions' between ePROs and clinicians' assessments were debated alongside strategies to use ePROs as complementary to clinicians' assessments (q45-47).
Variable use of ePRO data. Despite tailored educational support, use of ePROs was variable. In phase II, clinicians were invited every 2 weeks to anonymously answer two rating-scale questions. They did so a total of 244 times; 47% of the time they looked at ePRO results most or all the time, and 42% of the time they used ePRO information most or all the time (Figure 4). In summary, ePRO data were viewed or used less than half of the time.
Clinicians' knowledge and beliefs about use of ePROs, compared before and after workshops, showed disparate uptake: 35% of clinicians reported an increase in looking at ePROs, 57% improvement in skills explaining ePRO completion to a patient, 38% greater competence in follow-up, and 51% greater understanding that patients' self-reports differed from proxy reports. However, 33% found less importance for ePROs being used by kidney programs or practitioners, 25% less enhancement of person-centered care with routine ePRO use (64% no change), and a 35% decrease in clinician responsibility to respond to ePROs ( Figure 5). Volume 14 18 journals.sagepub.com/home/taj TherapeuTic advances in chronic disease After the workshops, clinicians' narratives about how they used ePROs was clinician dependent, spanning the spectrum of person-centered kidney care. On one end, clinicians did not change how they used ePROs (q48), did not consider them useful (q49), and no longer invited patients to complete ePROs if they were not participating in the study (q50). The majority of clinicians expressed increased awareness (q51), and looked at responses prior to a patient leaving the clinic to 'address it' (q52). On the other end of the spectrum, some felt their practice had changed and that ePROs facilitated their engagement with a person 'as a whole' (q53 and 54).
Patients did not know who saw their ePRO responses (q55), which were frequently not acknowledged (q56). Patients were left wondering if they could or should bring them up, or if that was the clinicians' responsibility (q57 and 58). Patients saw value in completing ePROs, appreciated seeing their trends over time, and envisioned ePROs supporting 'a mindful part of the patient and clinician experience' (q59) or selfcare (q61 and 62).

Discussion
We found that person-centered care was not enhanced through educational support for multidisciplinary kidney clinicians to routinely use ePROs. Overall PACIC score and domain scores were similar at implementation and non-implementation sites throughout the study, including after clinician workshops. While the extant PRO literature emphasizes training clinicians, 2,19-23 our findings show that this alone seems insufficient to show enhancement in person-centered care. The qualitative findings confirm the PACIC analysis, and offer explanatory insights for this departure; each of the three insights are discussed in relation to literature in the field.
First, clinicians wanted self-reports on kidney symptoms, not QOL. Similar to many chronic illnesses, treatment of symptoms and use of technology has reified 'dialysis-centered care'. 46 Health systems reinforce this by care being organized by body parts; clinicians specialize accordingly. Understandably, clinicians wanted to see what they were familiar with. They may also have wanted to see which kidney symptoms lowered In phase II, clinicians at the implementation site were asked to anonymously answer the below rating-scale questions. They were invited every 2 weeks to answer. Responses were anonymous; n = 244 denotes the number of submitted responses (215 responses from allied health care providers; 29 responses from nephrologists); Q1 had two missing; Q2 had five missing).  Clinicians who attended workshops were invited to complete a questionnaire regarding their knowledge and beliefs about use of ePROs. The first questionnaire was completed before they attended a workshop, and the second questionnaire was completed after all workshops were delivered. % increase = the increase % of respondents who provided an affirmative response to any of the questionnaire items. Questionnaire items: 1. Do you look at these PROs? 2. If you were asked to explain to a home dialysis patient why they were completing the patient-reported outcomes, and what was going to be done with their reports, how would you assess your own skills? 3. Today, how competent would you feel if you were asked to follow up with a home dialysis patient regarding their responses in their patient-reported outcomes? 4. It is very important for home dialysis programs and practitioners to use patient-reported outcomes. 5. Person-centered care in home dialysis can be enhanced by routine integration of patient-reported outcomes. 6. It is my responsibility to review and respond to a patient's quality of life concerns beyond those traditionally connected to kidney disease (i.e. pain, depression, anxiety). 7. It is reasonable to believe that a patient's reports of their health, quality of life, and experiences of care may change over time, from one clinic appointment to the next. 8. A patient's report of their symptoms, quality of life, and experiences of care would be reported exactly the same as if a healthcare practitioner reported on their behalf (i.e. in proxy). ePRO, electronic patient-reported outcome.
TherapeuTic advances in chronic disease dialysis-centered to person-centered. 9,49-51 Future patient-oriented research may focus on empowering the person receiving dialysis to routinely use ePROs, an approach not previously explored.
Third, clinicians' use of ePRO data was variable, only looking at and using it less than half of the time.
To address ongoing questions about use of ePROs in routine kidney care, we created two animated videos, one for clinicians and one for patients, which were made available after the study was completed (https://www.healthyqol. com/kidney). In the global movement toward person-centered kidney care, 1,48-51 use of PROs by clinicians may ultimately be less important than addressing systemic challenges in dialysis care that hinder person-centered practices, such as traditional provision of healthcare where clinicians' knowledge or biomedical information is privileged over patient knowledge, and healthcare organized in silos according to body parts (i.e. kidney, cardiac, mental health). 26 The strengths of our study include longitudinal ePRO collection in a large sample alongside a mixed methods approach for data triangulation with insights illuminated through strategic approaches to rigor, further informed by the Knowledge-to-Action Framework, 30 including credibility (i.e. iterative, co-development of an educational support with clinicians, use of multiple data sources, attention to negative case analysis); confirmability (i.e. reflexive field notes, audit trails during data collection and analysis, interviews with research assistants who collected data at the two sites); and transferability (tailoring workshops to local context while leveraging extant literature). Nevertheless, our study is not without weaknesses. Although clinicians received training on integration of ePROs into care, they elected to only receive an ePRO focused on symptoms. In addition, clinicians at the implementation site had previously collected this information for 8 years prior to the study, and it is unknown whether this impacted findings. An ePRO focused on symptoms may not have been the optimal information for clinicians to see, and it may have limited a person-centered approach to kidney care, which is intended to encompass the whole of a person. [9][10][11][12][13] The PACIC-20 showed that care was already very good in both sites, thus improvement may have been untenable. Furthermore, the lack of integration of the ePRO digital health platform with the electronic medical record system used by the clinicians at the implementation site may have impacted their receptiveness to routine use of ePROs. Finally, this study was conducted prior to the COVID pandemic. With the widescale adoption of telehealth options during the pandemic, future studies may investigate whether in/acceptability of telehealth innovations in kidney care impacts routine use of ePROs.

Conclusion
The results of this study reveal that training clinicians on the use of PROs is complex and likely only part of what is required to enhance personcentered care. While this is counter-culture to the extant literature that encourages clinician training for routine collection, use, and integration of ePROs, person-centered kidney care may necessitate a wider strategy that addresses the complexity of such a culture shift. Our findings provide an important contribution to the literature because we identified three insights to explain why this might have occurred. First, clinicians wanted selfreports on symptoms, not QOL. Second, workshops were tailored to clinicians' educational needs, not patients. Third, use of ePRO data by clinicians was variable. Our findings emphasize the need for research that emphasizes patients' priorities in using their self-report to inform their care. While ePRO use has the potential to improve patient outcomes, it seems to require intervention beyond clinician education.

Declarations
Ethics approval and consent to participate Ethical approval was granted by University of Alberta / Alberta Health Services (#Pro00068365) and the University of Calgary (#REB17-0506).
Informed written consent was obtained at the first point of data collection and re-confirmed at subsequent interactions.

Consent for publication
Not applicable.